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(57) ABSTRACT 

A method and system for integrating web photo-services for 
a browser-enabled device is disclosed. The method and 
system include providing a server that communicates with 
the device over a network and associating images stored on 
at least one photo-service site with a user account. There- 
after, an inventory of images stored on the device is received 
from the device, and an image-related web application is 
provided to the device over the network, where the web 
application requires access to the user's images. The method 
and system further include providing a list of the images 
associated with a user's account to the web application, 
wherein the list of images includes an image reference for 
each image and an indication of whether each image is 
stored on the device or on the photo-service site, such that 
the web application may perform at least one function on the 
user's images regardless of where the images are stored. 
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META-APPLICATION ARCHITECTURE FOR 
INTEGRATING PHOTO-SERVICE WEBSITES FOR 
BROWSER-ENABLED DEVICES 

CROSS-REFERENCE TO RELATED 
APPLICATION 

[0001] The present invention is related to co -pending U.S. 
patent application Ser. No. 09/698,777, entitled "Meta- 
Application Architecture For Integrating Photo-Service 
Websites filed on Oct. 27, 2000, and assigned to the 
assignee of the present invention, and hereby incorporated 
by reference. 

FIELD OF INVENTION 

[0002] The present invention relates to manipulating digi- 
tal images over the Internet, and more particularly to pro- 
viding an architecture for integrating photo-service-based 
websites for access by client devices. 

BACKGROUND 

[0003] As the popularity of digital cameras grows, the 
desire of digital camera users to share their images with 
others will also continue to grow. The best approaches to 
photo-sharing take advantage of the Internet. Several Inter- 
net companies now offer an even more convenient approach 
by providing photo-sharing websites that allow users to store 
their images for free and to arrange the images into web- 
based photo albums. Once posted on a photo-sharing web- 
site, others may view the images over the Internet. 

[0004] The assignee of the present invention has devel- 
oped a system for uploading images to the Internet, directly 
from the camera, as described in U.S. patent application Ser. 
No. entitled "Method And System For Automati- 
cally Configuring A Hand-Held Electronic Device For 
Accessing A Site On A Public Network" filed on Jul. 26, 
2000. 

[0005] In this system, cameras connect to a gateway server 
on the Internet via a service provider, which may include a 
wireless carrier and/or an Internet service provider (ISP). In 
order to create a camera that requires no configuration to 
connect to the Internet, the camera is provided with a 
software application that is pre-configured to establish com- 
munication with the ISP and the gateway server. Upon 
establishing a connection, the camera sends they user's 
account ID and password to the gateway server. The user 
account information is then stored on the camera for use the 
next time the electronic device accesses the website. Thus, 
the user does not have to enter account information in order 
to establish the ISP connection or the website account before 
accessing the Internet. 

[0006] The gateway performs two basic services for the 
client. First, it is the camera's home base, which provides 
authentication services (user and device) and configuration 
services (it updates the camera's configuration, so the user 
doesn't have to). Second, it receives and responds to the 
camera application's requests using a protocol both under- 
stand. Services available to a camera may include the ability 
to send images from the camera to a specific photo-service 
service and the ability to send emails with links to uploaded 
images. 

[0007] The current gateway solution is built on traditional 
client-server architecture, where a software application on 



the camera communicates with a software application on the 
server. Client-server architecture requires custom software 
on all three tiers of the current architecture; the camera, the 
gateway, and the photo-service site. In addition, the current 
gateway solution only enables communication with digital 
cameras, not other mobile devices. 

[0008] A newer model for application deployment on the 
Internet today is server-based (i.e., ASP model), where a 
client device equipped with a web browser communicates 
with a web server. Browser-based devices simply download 
web pages from the server, which provides the application 
function and data. The deployment of web applications 
using this new server-based architecture is growing much 
faster than the deployment of client-server based applica- 
tions because browser-based clients do not require a gate- 
way that "speaks" the client application's protocol. Thus, 
browser-based clients may connect to the photo -service sites 
directly, since the devices are browser-based. In addition, 
browser-based clients also do not require embedded custom 
software for requesting imaging services from the photo - 
service sites. Instead, once connected, users of these devices 
could interact with the photo-services sites directly using the 
device's browser to display web applications from the 
photo-service sites if the sites support the specific browsers 
in these devices, or indirectly via a transcoding gateway. 

[0009] A transcoding gateway converts the sites' HTML 
to a format suitable for the various browser types. Transcod- 
ing products exist today that can support multiple browser- 
based clients that are both wireless and wired. Transcoding 
technology takes a formatted input stream (typically HTML) 
from a web server and converts it to an output stream in 
another format (e.g., WML for WAP phones, cHTML for 
i-mode phones, etc) of a particular type of browser-based 
device. Digital cameras will soon be equipped with brows- 
ers, just as PDA's and cellphones are, and such transcoding 
products allow, or soon will allow, browser-based devices to 
access the images and image services of photo-services sites. 
[0010] There are two main problems with equipping digi- 
tal cameras with web browsers for communication with 
photo-service sites and for running their web applications. 
One problem is making the presentation of the web appli- 
cations palatable to the various types of browser-based 
devices, given the variety of display characteristics and 
browser technologies. There are two approaches to address- 
ing this presentation problem. 

[0011] One approach is for each photo-service site to build 
custom web pages for each specific device/browser type. 
That is, the photo-service site would need to provide web 
pages formatted in HTML, WML, cHTML, and so on, and 
preprocess images to suit the device display capabilities. 
This is both labor intensive to initially setup and difficult to 
maintain as changes are made to the site's data and services. 
[0012] A second approach is to use a transcoding product, 
such as a WAP gateway or Oracle's Portal -to-Go. The 
problem with the transcoder approach is that it tries to solve 
a very broad problem, making all HTML encoded informa- 
tion presentable in a number of other different formats. 
Consequently, transcoders often produce unsatisfactory 
results. Transcoders thus serve as a temporary solution while 
photo-service sites build support for each of the various 
devices directly into their sites. 

[0013] As digital imaging grows in popularity, there will 
be a need for disparate photo-service sites to integrate their 
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offerings (e.g., photo-hosting from one, and printing from 
another). This requires that two photo sites wishing to 
become partners must each enable their sites to communi- 
cate. Neither of the two approaches described above 
addresses the requirement of integrating the services that 
span the sites of multiple photo-service providers. Since 
there is no standard for inter-site communication for photo - 
service sites, this effort must be undertaken for each new 
partner a site agrees to work with. 

[0014] The second problem with equipping digital cam- 
eras with web browsers for displaying web applications 
from photo-service sites is the limitations inherit in web 
browsers, which is that browsers typically do not allow web 
applications to have access to content of the requesting 
device. Using a PC environment as an example, assume a 
user wants to upload images to a photo-sharing site on the 
Internet using a browser. To upload images, the user navi- 
gates to the photo sharing site and clicks an "upload" button. 
In response, the photo sharing site sends an upload web page 
to the user's PC. Because of the web browser does not allow 
the upload web page to access to the hard drive, the upload 
page displays several blank image name fields for the user 
to fill-in. If the user does not know the names of the images, 
the user must click a "browse" button on the web page in 
order to search the directories on the PC for the desired 
image files. Once the user navigates to the correct directory 
and selects one of the images files, the name of the image file 
is then inserted into one of the image name fields on the web 
page. The process is then repeated for each image the user 
wants to have uploaded. 

[0015] Due to limitations imposed by web browsers on 
web applications with respect to the ability to access the 
internal storage of the requesting device, the process of 
manipulating images over the Internet via web browsers is 
burdensome and inefficient. 

[0016] Accordingly, what is needed is a method and 
system for integrating web photo-services for browser- 
enabled client devices. The present invention addresses such 
a need. 

SUMMARY OF THE INVENTION 

[0017] The present invention provides a method and sys- 
tem for integrating web photo-services for a browser-en- 
abled device. The method and system include providing a 
server that communicates with the device over a network, 
and associating images stored on at least one photo-service 
site with a user account. Thereafter, an inventory of images 
stored on the device is received from the device, and an 
image-related web application is provided to the device over 
the network, where the web application requires access to 
the user's images. The method and system further include 
providing a list of the images associated with a user's 
account to the web application, wherein the list of images 
includes an image reference for each image and an indica- 
tion of whether each image is stored on the device or on the 
photo-service site, such that the web application may per- 
form at least one function on the user's images regardless of 
where the images are stored. 

[0018] According to the present invention, the function of 
the web application is extended by allowing the web appli- 
cation to have access to references to the user's images, but 
not to the images themselves. Thus, the present invention 



overcomes the limitations imposed on the web application 
by the web browser and allows the web application to make 
intelligent decisions about what functions to perform on the 
user's images regardless of the images' storage locations. 

DESCRIPTION OF THE DRAWINGS 

[0019] FIG. 1 is a block diagram illustrating a meta- 
application architecture for an online system in accordance 
with a preferred embodiment of the present invention. 

[0020] FIGS. 2A and 2B are a flow chart illustrating a 
process for allowing a web application to access image files 
stored on both a client device and distributed across remote 
locations in a preferred embodiment of the present inven- 
tion. 

[0021] FIG. 3 is a diagram illustrating an example image 
list sent from the gateway server to the web application 
contracting with the browser of the client device. 

[0022] FIG. 4 is a flow chart illustrating the process of 
uploading images from a web-enabled client device using a 
web application that is accessed through the image gateway 
in accordance with a preferred amount of the present inven- 
tion, 

DETAILED DESCRIPTION OF THE 
INVENTION 

[0023] The present invention relates to an online digital 
imaging architecture. The following description is presented 
to enable one of ordinary skill in the art to make and use the 
invention and is provided in the context of a patent appli- 
cation and its requirements. Various modifications to the 
preferred embodiment will be readily apparent to those 
skilled in the art and the generic principles herein may be 
applied to other embodiments. Thus, the present invention is 
not intended to be limited to the embodiments shown but is 
to be accorded the widest scope consistent with the prin- 
ciples and features described herein. 

[0024] The present invention provides a meta-application 
architecture for allowing photo-service websites to receive 
and send images to and from a wide range of client device 
types, and for integrating the services of the photo-service 
sites 14 for access by users of the client devices. The present 
invention also allows web applications running in a web 
browser on the client devices to access all of the user's 
image files regardless of whether the image files are stored 
on the client device or on sites on the Internet, thereby 
enhancing imaging services provided to the client devices. 

[0025] FIG. 1 is a block diagram illustrating a meta- 
application architecture for an online system 10 in accor- 
dance with a preferred embodiment of the present invention. 
The system 10 includes multiple client devices 12 that 
request imaging services from multiple online photo -service 
sites 14. The photo-service sites 14 are sites on the Internet 
that provide different types of digital imaging services. For 
example, one photo-service site 14 may provide an image 
hosting service, while another photo-service site 14 provides 
image printing services, for instance. 

[0026] A client device 12 refers to an electronic device 
capable of capturing and/or displaying digital images and 
communicating over a network, such as the Internet. Such 
electronic devices include devices that store digital images, 
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such as PCs and photo kiosks, and image capture devices 
such as digital cameras and PDAs, and cellphones that have 
lens attachments, for example. In a preferred embodiment, 
the client devices 12 are browser-based, although non- 
browser-based devices may also be supported. The client 
devices 12 communicate over the Internet via a wireless, or 
wired connection, but because they incorporate different 
browser types, the client devices 12 communicate data in 
different formats. For example, some client devices 12 such 
as PCs may communicate data in HTML format. Other client 
devices 12 such as cellphones, may use data formats such as 
Wireless Markup Language (WML), which is a streamlined 
version of HTML for small screen displays, cHTML which 
is a subset of HTML, and Handheld Device Markup Lan- 
guage (HDML), for instance. 

[0027] In one embodiment, the client devices 12 connect 
to the Internet via a service provider 16, which may include 
a wireless carrier and/or an Internet service provider (ISP). 
Once connected to the Internet, the client devices 12 have 
the capability of uploading the digital images to the online 
photo-service sites 14 for storage and/or for receiving digital 
images from the photo-service sites 14 for display. 

[0028] One aspect of the present invention provides a 
meta-application 22 architecture that provides a common 
communication framework for integrating photo-service 
sites 14 and services for client devices 12. The meta- 
application architecture includes a site on the Internet, 
referred to as the image gateway 18, that interfaces between 
the client devices 12 and the photo -service sites 14. In a 
preferred embodiment, the image gateway 18 includes a 
gateway server 20, a software meta-application 22, and a set 
of site adapter software 24 that provide a set of standard 
APIs and data formats that the photo-service sites 14 use so 
that the image gateway 18 can present data and services 
from the sites 14 to the various client devices 12. These same 
APIs and data formats allow the image gateway 18 to 
present the services of multiple photo-services 14 in one 
integrated application, and allow communication among the 
photo-services sites 14. For example, the image gateway 18 
enables a user with images stored on one photo-host site to 
access to the services of all print service providers who also 
use the image gateway 18. The photo-hosting site 14 would 
not need to make any special effort in order to work with the 
print service providers since they are all bound together by 
the meta-application. 

[0029] The gateway server 20 performs the function of a 
web server, while the meta-application 22 performs the 
function of an application server. The meta-application 22 
may reside on the same or different computer from the 
gateway server 20, and one of the photo-service sites may be 
part of the image gateway 18. The gateway server 20 
provides client device 12 connectivity and is primarily 
responsible for detecting the client device 12 type and its 
browser type and display characteristics. The gateway server 
20 may also provide security, configuration, and adminis- 
tration services, including the collection of usage statistics. 

[0030] In a preferred embodiment, the gateway server 20 
passes the data and service requests of the client devices 12 
and from the meta-application 22 in a device independent 
fashion. 

[0031] One reason that there is no standard for commu- 
nication between photo-service sites 14 is because each 



photo -service site 14 represents its own data and services in 
different formats. For example, all photo-hosting web sites 
14 organize a user's images in a nested tree-like structure 
similar to a file directory, but the names of the nodes in these 
trees vary across sites. For instance, some of the terms used 
include "album,""pholio,""page," and "shelves". 

[0032] According to the present invention, the meta-ap- 
plication 22 abstracts the underlying data model and the 
function provided by the photo-service sites 14, which is 
common across the photo-service sites 14, to define a 
common data model format for the data, referred to here as 
a meta photo-service model. In a preferred embodiment, the 
meta photo-service model is implemented using XML. 

[0033] Since each photo-service site 14 may use its own 
data model and define its own API or protocol for accessing 
the site's functionality, respective site adapters 24 are used 
to convert between the data and service formats of each 
photo-service site 14 and the meta photo-service model 26. 

[0034] In a preferred embodiment, the image gateway 18 
is provided with a database 32 for supporting the aggrega- 
tion of data and services across the various photo -service 
sites 14. This enables the image gateway 18 to support a 
single login for a particular client device 12 and enables data 
sharing, such as billing information, across photo -service 
sites 14. This data sharing eliminates the need for users to 
reenter this information for each site, but requires that the 
database 32 be synchronized with the data stored on the 
photo serving sites. 

[0035] According to a further aspect of the present inven- 
tion, developers who have registered with the image gate- 
way 18 may post web applications 42 on the image gateway 
18 for access by the client devices 12. In a preferred 
embodiment, the web applications 42 are imaging related 
and allow the users of the client devices 12 to manipulate 
their images in some manner. Examples of such im aging - 
related web applications 42 that may be provided include an 
upload image application that uploads images from the 
client device 12 to a photo-hosting service 14 via the image 
gateway 18, and a search application that searches for the 
user's images, for instance. 

[0036] It should be noted that although the terms images 
as used herein includes media types such as still images, 
burst images, and time lapse images, the term images also 
encompasses media types such as movies, sound annota- 
tions, animations, and clip art, for instance, 

[0037] In a preferred environment, the web applications 
42 are implemented as server-side processes that allow web 
pages to interact with databases and other applications. 
Examples of such server-side processes include active server 
pages (ASPs), CGI scripts and JavaServer Pages (JSPs), 
which are web pages that contains HTML and embedded 
programming code that is executed by a server. When a web 
browser makes a request from the web application 42 for a 
web page, the server executes the embedded program, and 
the HTML provides the page layout that will be returned to 
the web browser. The programming code provides the 
processing for the page, such as delivering search data 
entered on a web page to the database for lookup. It would 
also format the results of that search as HTML and send it 
back to the client device 12 for display, 

[0038] When a user attempts to work with his or her 
images on the browser-enabled client device 12 using one of 
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the web applications 42, the user's images may be stored in 
one of three ways; 1) on the client device 12, 2) on one or 
more photo-service sites 14, or 3) on both the client device 
12 and one or more photo-service sites 14. 

[0039] Displaying the user's images on the client device 
12 using a conventional browser may be accomplished in 
one of two ways. First, the images stored locally on the 
client device 12 could be displayed by storing an HTML 
page that references those images in the device 12 and then 
opening the HTML page in the web browser. Second, the 
images that are stored on a web server could be displayed by 
the web server by sending an HTML page referencing those 
images to the web browser on the device 12. Where the 
conventional browser fails is where the HTML page is being 
sent from the server to the device 12, but the image files that 
need to be referenced are stored on the client device 12. 
Thus, if the web application 42 needs to access images 
stored both in the device 12 and on the server, a problem 
arises because the browser on the client device 12 will 
typically not allow the web application 42 access to the 
contents of the client device 12. In addition, the web 
application 42 would have no way of knowing about the 
user's images that are stored on other photo-service sites 14. 

[0040] Besides providing a method for integrating web 
photo-services for a browser-enabled device, the present 
invention also allows a web application 42 sent from one 
server to know about files stored in locations other than that 
server. More specifically, the meta-application architecture 
of the present invention provides web applications 42 (under 
strict control and security) access to the user's images, 
which may be stored both locally on the client device 12 and 
distributed across photo-service sites 14. 

[0041] FIGS. 2A and 2B are a flow chart illustrating a 
process for allowing a web application 42 to access image 
files stored on both a client device and distributed across 
remote locations in a preferred embodiment of the present 
invention. The process begins by providing a gateway server 
20 that communicates with the client device 12 and associ- 
ates images from the client device 12 with a user account in 
step 100. As described above, the user's previously uploaded 
images may be distributed across various photo-service sites 
14. 

[0042] The client device 12 is also provided with software 
that is capable of reporting the image contents of the device 
to the gateway server 20 in step 102. The software may 
report the image contents of the device either automatically, 
or at the request of the user or the image gateway 18. In a 
preferred enlightenment of the present invention, the soft- 
ware that reports the image contents to the gateway server 20 
is a customized web browser. In an alternative embodiment, 
the underlying software in the client device 12 that estab- 
lishes the connection to the gateway server 20 is responsible 
for reporting the images in the client device 12 to the 
gateway server 20. In the second embodiment, the browser 
itself need not know about the images directly, but only 
through references provided via downloaded pages from the 
gateway server 20. 

[0043] Once communication between the client device 12 
and the image gateway 18 has been established, the client 
device displays a web page from the gateway server 20 
indicating what web applications 42 are available to user in 
step 104. In a preferred environment, the available web 



applications 42 are displayed via hyperlinks. For example, 
the web page displayed to the user may display links such as 
"Upload Images," and "Search For Images," which link to 
corresponding web applications 42. 

[0044] In response to the user selecting a web application, 
the gateway server 20 connects the client device with the 
selected web application 42 in step 106. Those with ordinary 
skill in the art will appreciate that the connection is prefer- 
ably established with a secure handshake mechanism. 

[0045] When the web browser in the client device 12 
begins interacting with the web application 42, the web 
application 42 sends a request to the gateway server 20 
asking what images are available for the user in step 108. In 
a preferred embodiment, the web application 42 identifies 
the user to the gateway server 20 using the user account or 
user ID, which was provided to the web application 42 when 
the connection was made to the application 42 by the 
gateway server 20. In response, the gateway server 20 
prepares and returns a list of image references and other 
information corresponding to the user's images in step 110. 

[0046] FIG. 3 is a diagram illustrating an example image 
list 50 sent from the gateway server 20 to the selected web 
application 42 interacting with the browser 54 on the client 
device 12 through the gateway server 20. In a preferred 
embodiment, the image references in the list 50 comprise 
image identifiers (IDs) 56 that uniquely identify each image. 
The image IDs 56 may comprise a number or a name, or an 
internal disk reference (e.g., file path). The information 
included in the list may include the location 58 of each 
image (e.g., the device or a server), and may even include 
information about which server. The information may also 
include metadata 60 corresponding to each image. The 
metadata 60 is data associated with an image that is either 
embedded within the image file or separately in a file or 
database. Examples of metadata 60 include values for 
parameters such as f-stop, zoom factor, focus distance, 
category tags, image name, camera manufacturer and model 
number, and so on. Specific metadata may be requested by 
the web application to be included in the list, including 
custom user metadata. 

[0047] According to the present invention, the web appli- 
cation 42 interacting with the browser on the client device 12 
is not given access directly to the user's images, instead the 
web application 42 is only given access to information about 
the images via the image list 50. The information in the list 
50 returned to the web application 42 is sufficient to allow 
the web application 42 to sort and select the images to carry 
out its function. 

[0048] Referring again to FIGS. 2A and 2B, after receiv- 
ing the image list 50, the web application 42 selects a set of 
images to reference for display and/or to perform a function 
on in step 112. As shown in FIG. 3, the web application 42 
places these references in web pages that arc returned to the 
device browser 54 through the gateway server 20. 

[0049] Referring again to FIG. 2A, for images that are 
identified in the list 50 as being stored locally on the client 
device 12 in step 114, the web application 42 generates a 
reference that comprises a file path or other pointer to the 
image in the client device 12 along with a resize command 
in step 116. Preferably, this translation from image ID to the 
file path is performed by the gateway server 20 when the 
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web page containing the ID passes through on its way from 
the web application 42 to the client device 12. 

[0050] For images that are identified in the list 50 as being 
stored on a photo-service site 14 or other server in step 118, 
the web application 42 makes a request for the image from 
the gateway server 20 using the image ID in step 120. The 
gateway server 20 then fetches the image from the indicated 
location, resizes and converts the image to the required 
format, and passes a URL to the resulting resized image file 
back to the web application 42 in step 122. The web 
application 42 then inserts this URL into to the web page that 
is transmitted to the device browser 54 in step 124. Alter- 
natively, the translation from image ID to a URL to a resized, 
converted image file is performed at the gateway server 20 
when the web page containing the ID passes through on its 
way from the web application 42 to the client 12. application 
42 For image viewing, there is no actual requirement for the 
web application 4242 to have a copy of the image or images 
being displayed on the client device 12. 

[0051] The entire viewing function can be done with 
image references, along with appropriate gateway functions 
and browser requests for local files. However, when a web 
application 42 wishes to perform an operation on an image, 
such as color balance, contrast enhancement, rotate, etc., an 
actual copy of the image is required at the web application 
4242. Thus, the web application can request a copy of the 
image from the gateway server 20 in any resolution up to full 
image resolution. In this case, the web application 4242 will 
perform whatever function desired, and create any versions 
of the image, such as thumbnails, within its own file system 
for reference by the browser 54. The modified images can be 
temporary or permanent. For permanent images, the web 
application 4242 must request that the gateway server 20 
store the resulting image in an appropriate location, depend- 
ing on the user's account information. 

[0052] The web application 42 may also need to delete 
images selected by the user. For example, if a modification 
is performed, and the user wants to keep the modification 
and not the original, a delete function is required. If images 
arc uploaded from the client device 12 to a photo hosting/ 
sharing service, the user may wish the copies in the device 
12 be deleted, thus eliminating duplicate storage. However, 
allowing the web application 42 to delete images is a 
dangerous practice. It is assumed that all destructive opera- 
tions are carried out over secure connections to prevent 
unauthorized access, but even with this protection, addi- 
tional security protection is required. 

[0053] The preferred embodiment is for the delete func- 
tion to be handled by the gateway server 20. In this case, any 
image delete functions must be requested by the web appli- 
cation 4242. The gateway server 20 would be responsible for 
issuing the appropriate warning to the user via the browser 
54 or its underlying software. Additionally, the gateway 
server 20 may cache copies of all deleted files for a period 
of time or until the user "empties the trash," thus preventing 
the user from accidentally destroying valuable images. This 
is especially true for deletes of original images when image 
modifications are done. It is good practice to never delete the 
original image, and carry modifications via additional files. 

[0054] The present invention will now be explained by 
way of a particular example where the web application 42 
provides an image upload function for the user of the 
browser-enabled client device 12. 



[0055] FIG. 4 is a flow chart illustrating the process of 
uploading images from a web-enabled client device 12 using 
a web application 42 that is accessed through the image 
gateway 20 in accordance with a preferred embodiment of 
the present invention. During normal operation of the client 
device 12, the user may be shown a homepage of the image 
gateway 20, which displays a selection of image-related 
functions the user may want to use in step 200. In response 
to user selecting the choice to upload images to a photo - 
service site 14, the gateway server 20 connects the client 
device 12 to a corresponding upload web application 42 in 
step 202. The gateway server also provides the upload web 
application 42 with the image list 50 identifying the user's 
images in step 204. As described above, the image list 50 
will identify both images present in the client device 12 as 
well as images stored on the client device and the image 
gateway 18 and other photo service sites 14. 

[0056] The upload web application 42 will then automati- 
cally select images stored on the client device from the 
image list 50 and present corresponding thumbnail images to 
the client browser via HTML tags in step 206. In a preferred 
environment, the HTML tags incorporate the image IDs 
and/or file paths from the image list 50 and may also include 
height and width tags for resizing the original image into the 
thumbnail image, 

[0057] The browser 54 on the client device 12 then 
interprets the HTML and renders the images presented by 
the web application 42 (resizing the original images if 
necessary) in step 208. Since the images are local, no web 
traffic is required to service the image tags — they are 
accessed locally and resized locally. The web applications 
52 allows the user to select which of the displayed images 
to upload in step 210. In response, the web application 42 
uploads the selected images to the gateway server 20 in step 
212, where they are then transmitted on to the selected 
photo-service site 14. 

[0058] After successful completion of the upload, the web 
application 42 may ask the user if the device resident copies 
of the uploaded images should be deleted. If the user selects 
YES, a request for deletion is issued by the web application 
42 to the gateway sever 20. The gateway server 20 performs 
the appropriate deletion function, typically including getting 
confirmation from the user before proceeding. The confir- 
mation may come from the gateway server 20, or may come 
from underlying software in the client device 20, which is 
designed to intercept any delete requests from the browser or 
internet connection. 

[0059] As a further example, assume that a developer 
provides a search web application 42, which when run on the 
browser 54 of the client device 12 allows the user to enter 
search criteria, and in response, the search web application 
42 returns and displays a set of images meeting those 
criteria. In operation, the gateway server 20 would provide 
the web application 42 with a list 50 of the user's images. As 
described above, the image list 50 would include for each 
image an image ID 56, the location 58 of the image, and any 
metadata 60 associated with the image. The metadata 60 
could be specifically requested, based on the user criteria. 
The web application 42 would then use this information to 
find the images matching the user's search criteria. For the 
found images that are stored in locations other than the client 
device 12, the web application 42 would request that the 
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gateway server 20 fetch, resize, and convert these images for 
access by the browser 54 in the client device 12. The web 
application 42 would thus combine the images found on the 
client device 12 with the images transmitted from the 
gateway server 20 and display them to the user through the 
browser 54. 

[0060] According to the present invention, the meta-ap- 
plication architecture provides a service that extends the 
functionality of web applications 52 that function through 
the browser on the client device. Image-related web appli- 
cations 52 can now operate on all of a user's images without 
regard to where the images are stored and can make intel- 
ligent services available to the user. The intelligence for 
handling where the images are located and what to do with 
the images to make them display in the client device 12 is 
performed by the image gateway 18 for the web application 
42. Since most photo-service sites 14 today do not have the 
ability to interact directly over the Internet with client 
devices 12, the present invention provides a service that 
allows an interface designed for the LCD screen of the client 
device 12 to access the photo-service sites 14 that don't have 
that capability, and brings all the user's images under one 
service and one access point. 

[0061] Although the present invention has been described 
in accordance with the embodiments shown, one of ordinary 
skill in the art will readily recognize that there could be 
variations to the embodiments and those variations would be 
within the spirit and scope of the present invention. Accord- 
ingly, many modifications may be made by one of ordinary 
skill in the art without departing from the spirit and scope of 
the appended claims. 

What is claimed is: 

1 A method for integrating web photo-services for a 
browser-enabled device, comprising the steps of: 

(a) providing a server that communicates with the device 
over a network; 

(b) associating images stored on at least one photo-service 
site with a user account; 

(c) receiving from the device an inventory of images 
stored on the device; 

(d) providing an image-related web application to the 
device over the network, the web application requiring 
access to the user's images; and 

(c) providing a list of the images associated with a user's 
account to the web application, wherein the list of 
images includes an image reference for each image and 
an indication of whether each image is stored on the 
device or on the photo-service site, such that the web 
application may perform at least one function on the 
users images regardless of where the images are stored. 

2 The method of claim 1 further including the step of 
allowing the user to select the web application from the 
browser-enabled device. 

3 The method of claim 2 further including the step of: 

for images that are identified in the list as being stored 
locally on the browser-enabled device, generating by 
the web application a reference that comprises a file 
path to the image in the browser-enabled device along 
with a resize command; and 



for images that are identified in the list as being stored on 
a remote server, requesting a reference to a resized 
image from the server. 

4 The method of claim 3 further including the steps of: 

for images that are identified in the list as being stored on 
a remote server, configuring the server to fetch the 
image from the indicated location; 

resizing and converting the image to the required format, 
and 

passing a URL to the resized and converted image back to 
the web application for insertion into a web page that 
is transmitted from the web application to the browser- 
enabled device. 

5 The method of claim 4 further including the step of 
interpreting by the web browser the references from the web 
application and rendering the images on the browser-en- 
abled device. 

6 The method of claim 5 further including the steps of: 

allowing by the web application the user to select from 
among the displayed images; and 

performing a function on the selected images. 

7 The method of claim 6 further including the step of 
providing a customized web browser for reporting the 
content of the browser-enabled device to the server. 

8 The method of claim 7 further including the step of 
configuring the customized browser to report the image 
contents of the browser-enabled device automatically. 

9 The method of claim 7 further including the step of 
configuring the customized browser to report the image 
contents of the browser-enabled device by request. 

10 The method of claim 9 further including the step of 
showing available web applications to the user via hyper- 
links on a web page. 

11 The method of claim 10 further including the step of 
including metadata for each image in the list sent from the 
server to the web application. 

12 A system for integrating web photo-services for a 
browser-enabled device comprising: 

means for providing a server that communicates with the 
device over a network; 

means for associating images stored on at least one 
photo -service site with a user account; 

means for receiving from the device an inventory of 
images stored on the device; 

means for providing an image-related web application to 
the device over the network, the web application requir- 
ing access to the user's images; and 

means for providing a list of the images associated with 
a user's account to the web application, wherein the list 
of images includes an image reference for each image 
and an indication of whether each image is stored on 
the device or on the photo-service site, such that the 
web application may perform at least one function on 
the users images regardless of where the images are 
stored. 

13 The system of claim 12 wherein the user selects a web 
application from the browser-enabled device. 

14 'ITie system of claim 13 wherein for images that are 
identified in the list as being stored locally on the browser- 
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enabled device, the web application generates a reference 
that comprises a file path to the image in the browser- 
enabled device along with a resize command, and for images 
that are identified in the list as being stored on a remote 
server, the web application requests these images from the 
server. 

15 The system of claim 14 wherein the server further 
functions to; 

for images that are identified in the list as being stored on 
a remote server, the server fetches the image from the 
indicated location; 

resizes and converts the image to the required format, and 

passes a URL to the resized and converted image back to 
the web application for insertion into a web page that 
is transmitted from the web application to the browser- 
enabled device. 

16 The system of claim 15 wherein the web browser 
interprets the references from the web application and 
renders the images on the browser-enabled device. 

17 The system of claim 16 the web application allows the 
user to select from among the displayed images and per- 
forms a function on the selected images. 

18 The system of claim 17 wherein the web browser is 
configured to report the image contents of the browser- 
enabled device automatically, 

19 The system of claim 18 wherein the web browser is 
configured to report the image contents of the browser- 
enabled device by request. 

20 The system of claim 19 wherein available web appli- 
cations are shown to the user via hyperlinks on a web page. 

21 The system of claim 20 wherein metadata for each 
image is included in the list sent from the server to the web 
application. 

22 An online system comprising: 

a client device having user files stored thereon; 

a server in communication with the client device over a 
network, the server for associating files uploaded from 
the client device with a particular user and for trans- 
mitting the user files to a remote server for storage, the 
server further for supplying a web application to the 
client device; and 

a browser running on the client device for interacting with 
the web application, wherein the browser is customized 
to report the user files stored on the client device to the 
server, 

wherein when requested, the second server provides the 
web application with a list identifying the user's files 
that arc stored on both the client device and the remote 
server, such that the web application may operate on the 
all of the user's files regardless of the files* storage 
location. 

23 The online system of claim 22 wherein the files 
comprise image files. 

24 The online system of claim 23 wherein the web 
application performs imaging related function on the image 
files. 

25 The online system of claim 24 wherein the remote 
server comprises a web site separate from the server. 

26 The online system of claim 25 wherein the client 
device comprises an image capture device. 



27 The online system of claim 26 including a plurality of 
client devices that communicate data in different formats, 
and a plurality of online photo service sites, wherein each of 
the photo service sites utilize different data models, the 
server further including; 

a meta-application for defining a common data model 
format for the different formats of the photo service 
sites, such that when a request is received from a client 
device for photo services from a particular photo ser- 
vice site, the request is passed to the photo service site, 
and wherein when a response from the photo service 
site is received, the response is converted from the data 
model of the photo service site to the common data 
model format, the converted request is then presented 
to the requesting client device in the data format 
required by the requesting client device. 

28 A method for allowing a web application to access 
image files stored on both a client device and distributed 
across remote locations, the method comprising the steps of: 

(a) providing a gateway server that communicates with 
the client device and associates images from the client 
device with a user account; 

(b) providing the client device with a customized browser 
that is capable of reporting the image contents of the 
device to the gateway server; 

(c) connecting the client device with a web application; 

(d) receiving a request by the gateway server from the 
web application asking what images are available for 
the user; and 

(e) sending to the web application from the gateway 
server a list of image references corresponding to the 
user's images, wherein each image reference identifies 
one of the user's images and whether the image is 
stored on the client device or on a remote server, such 
that the image references allow the web application to 
sort and select the images to carry out a predefined 
function even though the web application is not given 
access directly to the user's images. 

29 The method of claim 28 further including the step of 
allowing the user to select a web application from the client 
device. 

30 The method of claim 29 further including the step of: 

for images that are identified in the list as being stored 
locally on the client device, generating by the web 
application a reference that comprises a file path to the 
image in the client device along with a resize com- 
mand; and 

for images that are identified in the list as being stored on 
a remote server, requesting the image from the gateway 
server. 

31 The method of claim 30 further including the steps of: 

configuring the gateway server to fetch the image from 
the indicated location; 

resizing and scaling the image; 

translating the reference to the image from a HTTP URL 
into a file path on the client device; and 

transmiting the resized image and its reference to the 
client device for use by the web application. 
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32 The method of claim 31 further including the step of 
interpreting by the browser the references from the web 
application and rendering the images on the client device. 

33 The method of claim 32 further including the steps of: 

allowing by the web application the user to select from 
among the displayed images; and 

performing a function on the selected images. 

34 The method of claim 33 further including the step of 
configuring the customized browser to report the image 
contents of the client device automatically. 



35 The method of claim 34 further including the step of 
configuring the customized browser to report the image 
contents of the client device by request. 

36 The method of claim 35 further including the step of 
showing available web applications to the user via hyper- 
links on a web page. 

37 The method of claim 36 further including the step of 
including metadata for each image in the list sent from the 
gateway server to the web application. 

***** 
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